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a These standards are based on the direct ‘experiences of 
| the staff of C.M. celauand Associates in archiving large numbers 

of data files and on the experiences others working in data _ ‘ 
archiving have shared with us. We have. attempted to “synthesize 
the efforts of pene people working in- the field into a single - 
comprehensive ssclegtes AK large sarin of the file-level documentation gs oS 
standard is based on the work of Dr. Richard Roistachér and Barbara.’ 

» Noble of the’ Bureau of Social Science Research. “Additional in-»’ . 


put was provided by Alice Robbin of the University of Wisconsig.and 


¢ 


* by many other researchers active in the data archiving field. 

We view these standards as vital archiving éesis and we 
view archiving as a service which will stimulate future researchers 
to conduct secondary analyses, thus maximizing the potential use of 
existing data files. We ‘hope that, as. experience with both Pne 
documentation standard on with archiving in general Sreus, the 4 ‘a 3 
standard will, evolve to greater usefulness. We ‘encourage 1 users of | | 
the standards to comment on their mupreienoes with the standards. ~ » o¢ 
Their comments will had b. M. Leinwand Associates, Inc. in extending , 4 


the greed of the standard to all types of data’ files. ~“ :* ‘ 
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.The following documentation standards were developed to provide data 


archivists with a consistent documentation standard rere applicable to 


small, single-survey data sets, as well as large evaluation projects in- 


volving hundred of related data files. The standards were developed to Pi 
FS en eg Ste Oe ee ree BAG aR 
. be all. clusive, and to explain fully the documentation necessary to describe 


. any ‘collection of data files. In using,the standards, data archivists will 


select those sections of the standard applicable to bheir specific documentation 
needs. In this way, consistent documentation can be developed for a widey 
‘range of projects. Two levels of’ documentation are described: project and 


file. | - 7  # 
‘ey ‘ \ : 
-Praject-level ‘documentation is a single volume designed to acquaint 
a potential data user’ with the research ‘and/or evaluative effort. that generated 
the archived data. It does this by providing background information ‘on an 
i? = i oN ny 
« 


yo. : ar ae 
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entire project, including ate original purpose, ee perspective, and 


enabling Apetalatign, Additionally, project-level documentation describes 


each of the ‘Ssubstudies undertaken as part of the project, detailing target * 6 


eee types of information contained in the data, sets, and key +aneings : 


J ) 
of ‘previous analyses. An interested researcher will read the project-level 


» 
documentation first to forn an initdal opinion as to meetin the inbamals 


. 
° : - . 
t 


serve his or her research needs. 


‘ 


File-leval documentation may consist of many volumes, each providing 


detailed information on a data set obtained from a-project. It specifies é 


élements of the individual files (e.g., unit of sheervabten. aule, data 
time frame) which can vary from file to file. ,File-level doc tation pre- 
sents additional information, such as RSTOOnO describing actual contents 
of the file»,and detadied descriptions of he universe, target, and obtained . 
populations. File-level detuned axoxion the individual data = aa 


- 


and serve ‘as the analyst's guide to their-use. File-level aipandionn sertutn 
a : . ie | 


extended Bib NT oECa pts of related documents, detailec coding and editing 


descriptions, and copies of the original data collection aE SPUBEHES For 


_ some projects, a separate varus describes each file; for other projects," 


several files are described in one volume or grouped together by substudy. ‘ 


Ne % 
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PROJECT -LEVEL DOCUMENTATION :STANDARD es 
: a . a : , .: 
. T.. INTRODUCTION : 


’ ST 


Traditionally, the documéntation of machine-readable data files has 


been limited to*""codehooks" or "record layouts" which provided basic 


, os 
, information about a data file's format, and content. Such documentation 


& 


’ \* ; : , 
‘ described each data field within, the ‘file, its SOEPEaOMs and its values. 


r 


or some, ens this limited documentation was ae for most ‘secoridary 


a 
4 ‘ 


eeaeeaeah ‘ee pbeei: The "codebook" and "record iayouts" bear at least, 


* some ‘regemblance to certain eapects ‘of the standard Proposed to create 
fiie“level dis peat seks (Volume IV of this series). Geiid: Aven 
) researchers seek information abdut the study which has generated ene” ) 
} data files, they are ‘esis réferred to the final repor® and oe 
the original’ proposal. A very ambitious, researcher can try to obtain 
‘ether study documents if they exist, suchhas, quarterly or progress 
reports, ‘memos, correspondence, and study notes. In addition, the 
origins broject staff can be interviewed when information gaps exist. 
. With much effort, the sSaburchien can: piece: together a fairly good ~ 
description of the study and Roetuiiy document any deviations from 
the original proposal over the course of the study. aan 


/ . ms Sa - 
Howevety this method is time consuming and costly and can fail to . 


a 


éprovide all of the information needed especially if the researcher was re 


relying only on coe eae Moreover, every researcher interested © - 


< 
in a study and its data would have to go through the’ same process. 


. 
- . 
(a * 
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‘As time elapses after the completion of a study, the supplementary 


materials become scarce and the original contractors engage in-other 


projects. Their memories of the earlier study fade and their patience 


-~ 
. 
‘ ° 


with repeated questions from 
‘ ; 


secondary analysts a sce ea 


The work of a secondary ‘analyst can be aided if a single document — 


already exists which accurately describes the research study. The 


quajity of See analytic studies can be improved if: researchers 
are spared the needless effort of reconstructing the original’ study. 
_This volume devoted to a Project-Level Documentation standard 


is a guide to the preparation of such documentation. 


, 


A. PURPOSE » : eo 


Prigject=levet Aceunddiatiioncaaoris as ,both an introdu¢tion to and over- 
view of a collection of lad files.. Descriptions uf component. data aia: \ 
narratives on the’ background and significance of tie data, and summaries - 
.of research goals and findings rapidly acquaint the reader with the key .- 
“eontiants and key concepts of the ne files. ‘Readers will use the documentation 


© * s 
_to determine if information contained jn the collected data files is potentially 


e a 


useful to them. 

Project-level documentation begins with Sections explaining the purpose, 

. , 

background, \and significance of the data files. Subsequent “sections deal 
with’ the study design, the sample represented by the data, the ergbe race 
analyses performed, the major sinesige of the study, and an overview of 
the actual Wata files aenerabed by the study. A full eassoueies of aes 
. files and their characteristic in file-level documentation and is presented 


, in a gocument separate from the.project-level documentation. Volume. IV 


in this series presents the standard for creating file-lével documentation. 


‘ 
« 
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mi, 
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B. ORGANIZATION OF THE PROJECT-LEVEL DOCUMENT 


° ‘ 8 


. ’ , , . . 
The project-level document has three major sections: project overview, 
. ~ 


v 
\ 


substudy’description, and ‘appendices. , 
io 
e The title ~ - 


’ 


é - fb 
@ The project overview summarizes the most important facts about 


the project and its historical significance. 

‘ | - a , Sao | Fe 
a 

, e The substudy descriptions include sections on substudy purposes, 


‘ 


findings, samples, and information on the contents of the data 


sets in the’substudy. 


’ 


@ The appendices inelude a glossary, bibliography, and other ' . “f 


, related materials. 
“ * 


II, PROJECT OVERVIEW og Pade 


The aaa consideration to be made before constructing the project 
is to.select a clear and meaningful title to identify the research project. 
The title iy be facntion’ to the analysis sence title in many cases. 
However, if such titles contain acronyms, Seca. or highly technical 
terns, they should be renamed since sich titles can mislead the reader. | aor 
The title, of course, appears on the title page and thus becomes the 
— reader's pine source of information about the documentatiormitself, the ° 
data being documented, ‘and the originators of both. A criticai issue ta 
naming a document pertains to access. . A title must contain sufficient infor- 
mation to permit uitealtane ave archivists to catalogue the data dad dee 
associated documentation and to hulp Yesdarchere aunte what has been archtved. 
‘The title of the project and its archive should be informative and concise, 
giving the researcher a highly’ compact description in a minimum of 25 words. 
Sometimes naming is easy, espeetaliy if a project title has been previously 
dcesinoet and consistently used iri.all project reports. In such cases, that 
title should also be used for the project-level documentation. If a title 


has not been created, then one should be developed incorporating the following 


components: 


- fe 


2 


e descriptive words indicating content; 
a ; ° e geographic focus or unit; 


e chronological year(s) of data target or data collection; 


e source of data (e.g., court records); 


: ' e study or series number (important if data files are ordered). oe, 
; ‘Example: = — : . 
i " The American\ National Election Study : ‘ ar 
(CPS Study 495401 °-- ICPR Study 7010) j Soa 
Pre-Election Wave -- Sept.1, 1972 --.Nov.6, 1972 : ; 


Post-Election Wave -- Nov. 7,’ 1972 -- Feb. 43, 1973" 


¥. 


a | 5 rn re 


. a 
4 


Example: 


"Change in U.S. Women's Sex Role atéabades WgeH= 1974 
#RO1-MH25271 . 
The National Institute of Mental Health" 


Acronyms do not describe a data file and can confuse unfamiliar 
_users--they should not be used in titles. For example, the "National 


a 


Longitudinal Surveys of Labor Market History" and the "National Longitudinal 


> 


. Study of the High, School Class ,of 1972" bight both be designated "NLS," 
a name whch lacks iabpdaska meaning as well as precise meaning. 
+) : To facilitate filing, and locating a title,-. begin the title with the os 
most descriptive word, not a number ae GR artiels (é.g., "An," "The"). ¢ 


A prefered title for the National Institute of Education Safe School ase 


~ Study is: aoe - ie 


“Safe School Study: Project-Level Dbcumentation," not 
‘ ae hs safe School Study: .....;" or 

"The Safe School Study: seeey™ OF 

"Project-Level Documentation for the Safe School study." 

Like the title, the title page initially identifies the dedumanbatien. 
It elaborates slightly on the information conveyed in the title by providings 

e@ authorship; . | Ta 

> 
e@e title of the document; , . 
e title lof the data bas be Mts documented; . 
: . 


* gas aicnenoeaie 


e imprint, including date(s), name(s), and: places ay: of the 
documentor and data collector; a 


e series statement. 


Figure lis a sample title page. 


. _ ¢@ 
ea ; Figure 1: Sample, Title sPage 
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, JUVENTic VETEN TION AND CCRRECTIGNAL FACILITY CohSuS OF 19718 
USER'S GulMe FOR T4t 4ACHINE nesurdie CAA Fal& <T> 


T . 4 
i] Te ee See = ig - ’ 
Produced by ar 
* 4 
0% . 4 UeSe Furcau cf the Census <3> 
~ ; ‘ Wdashinatons Dele 
* ; 1971 <o> 
for ; 


Nateonal Criminal yustice Information and Statistics Service ¢4> 
Law Enfogcetent 4sSsistance paninistration ) 
UeSe Oeodarrment of JusScice 
Washinrstons ceCe 2537 <5> 


@ 


~ 


4 
Reve Lead 1975 eaoe <P> 
sabieed Dy. LESA tata Archive ano Feseerch Suvcort Center <B> 
- Center for Advanced Cor outetion 
University cf Illinois 
Urbéenas IL 618061 s 
* €217). 225-3235 : 


-— 


User's Guide Preoared vy : 
LEAA Data Archive ang Reseerch Svud00rt Lénter <¥9> 
. ; CUnder LELA brant 77-35-S9-00NF) ; 
Decerber 1978 <10>_ a. 


a * ° q 
LEAA User's uigh Lth Ge <7> 


-! 
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The first alba of the project-level document is an overview of 
: Be. 

the important facts of the study and is designed to give the reader a 
thorough grasp of’ its subs$ance and evolution. These facts are organized 


thematically: fee 


) e@ Abstract : 
; @ Background - historical perspective and significance, issues 
addressed resultingein the undertaking of the study 
e Research -topits Foveukigabad , 
In making decisions about how much detail whala: appear in the project 
“ overview, it is helpful for archivists to remember that the overyiew is 
usually the first contact the potential user has with the data archive and, 
in most stu: with She abudlies themselves. The archivist should not assume 
that users have any previous knowledge about the study and wiouid give the 
’ reader all information necessary for an understanding of the study's basic 
purpose and structure. , 
m. ‘ + The intent of the project overview is to convey clearly and immediately 
the important elements of. the study, how these elements were conceived, and 
how they articulate with each other. Thus, the overview not only describes 
« _ the study's historical and theoretical background and the topics it in- el 
a vestigated, ee clarifies the overall coherence of these Sinaanes, Lies 
’ how they leat ents Piowed together. to form the study. : 
The length of the project overview dubded. with the complexity and scope | 
of the study. It is recommended that descriptions of complex studies re- 


quiring a lengthy overview employ subtitles for organization and emphasis. 


- 
@ 


(See example below. ) 


A. ABSTRACT | ' . | 
: The ep ge aumeraeer the project's subject matter, research 
5 ‘goals, data SOnESHt) and major findings. In a page or less, .it helps the 

reader to survey the contents of the study quickly and to ascertain whether 


the archive may contain data~of interest. It ‘should be short, informative, 


. * 
and easily comprehended by the reader. 


> \ 
; . ‘ »® 6 
EXAMPLE: Abstract 
The Safe School Study was undertaken by the National - 
Institute of Education in response to Congress’ re-— . 
quest that HEW determine the number of schools affectéd — 
‘by crime or’ violence, the type and seriousness of those 
crimes, and how school crime can be prevented. The 


study is based on a mail survey, of over 4,000 schools 
and on an on-site survey of 642 schools, and ease 
studies of 10 schools. Principals, teachers; .and 
stu@ents contributed to the ee 


Risks of chime at School a 


Although school violence and vandalism increased * 4S 
during the 1960's, they have leveled off since the, | 
early 1970's, and there are some' hints of a decline. 
Still, about 8%. of the nation's schools (6,700) 
have a serious problem with crime. Secondary , 
schools are more likely to have a serious problem 
than elementary schools. 
The risks of crime directed against schools are 
{ higher in the Northwest and West than in the North 
‘ Central and Southern States, and tend to be spread 
throughout urban and suburban areas. The risks of 
personal violence are higher in junior high schools . 
than in senior highs, and are higher in larger * 
communities. 


Extent of the Problem: Personal Violence 


About 2.4 million secondary school ‘students (11%) 
have something stolen from them in a typical month. 
About 1.3% of the students (282,000) report 

being attacked in a month. Relatively few are in- 
jured seriously enough to need medical attention. 


45 


? 


Among secondary school teacheps, about 12% (130,000) . 
have something stolen’ at school in a month's time, ‘ . atid be 
Some 52,000 are physically attacked, about 1,000 - 
of whom are seriously enough injured to require, 

medical attention. Around 6,000 have something . *, . 
taken from them by force, weapons, or threats. ; : 


- Young teenagers in cities run a greater risk of wt 
violence in school thah elsewhere, except in high - >" 

.  ¢rime neighborhoods. There, schools are safer than 

iy ‘the surrounding communities. ; - a 


. 


B. ' BACKGROUND. INFORMATION 
, Central to this section of the project overview is the rationale for ‘ 


collecting: the data being documented. After initially eee os purpose(s) 


‘ .) 


for the data collection effort: and the major research question(s), the 


historical context of the study is described. : * 
¢ 


Many projects are undertaken as part of an evaluation of a social ., 


’ program. In such cases, it is important that the reader understand the, 
0 ontext in which the program was conducted, as well as the specifics of the 
data themselves. Therefore, <the program is described--its goals, intended 
femuite, and the events leading to its ardwnmanettield: 
The. significance of the study and its‘data = be wentitad by reference 
to the research community and existing theory. Theoretical significance 
can be shown by answering the questions, | 
e What makes the data contained in this study of interest? 
e What is the nuintlouaniy of Suess data to those of other - 


' studies examining similar questions? ) \ 


e@ On what specific problems’ and issues did the stutly focus? . 


- 


Finally, if the collection of these data was precipitated by some 


key ae Mastin, such as a Congressional mandate, or in response to a 


riot, this seminal event or action is described. 


EXAMPLE: Background 

- The National Crime Surveys are conducted for. the Law , 7 
Enforcement Assistance Administration by the U.S. cers ; 

of the Census jw order to obtain current and reliableU: ' 

* , measures of serious: crime in the United States. The 
program is uniqve in that it turns to the public for : { 
information about crime rather than relying on law en- 
forcement records. The primary imp for this type 
of approach was an earlier’ victim sumfey which in- 
dicated that a considerable number of™erimes were not 
being reported and consequently existing. crime measures 
did not reflect the true rate of crime occurrence. In . 
addition to providing an alternative or supplemeritary Yq 


“ 


» source of data for estimating the amdfnt of crime, the ‘ 
surveys offer a level of descriptive detail about the | j P 
victims of crime and their victimization experiences 
not previously available. 
C. MAJOR RESEARCH QUESTIONS =< | i. 0 

The purpose of this section is to outline the major research question(s) We 
asked. Although this section could be as brief as a paragraph, it should 
not exceed three pages. are . 

® : aw. . 

It is important to note that while every study is guided by one 
or.more research questions, the level of specificity.may differ. soa 
often pose a general researgh question which is answered by posing “a 
multiple questions, each focusing on components of the larger question. 


‘ : > » 
This is particulaly true when overviewing a large-scale project which 


represents multiple substudies, eaci: of which,,in turn, entails several 


‘data Site eet Wherever possible, it is useful to choose * 


/ 


relatively specific set of questions for inclusion in- the overview, since 


this type of question is more meaningful to the reader. However, in the 


| 18 | as 


A 


. 


s 


, % so . 4 ; 
case of large-scale projects, this approach might result in pages of detailed. 
questions;, the archivist is then urged to chooséga smallen, number of more 


general questions. 


EXAMPLE: RESEARCH. QUESTIONS 


r 


6 . 
The ,Safe School Study was designed to ' : 
, address the following research questions: 


e@ How serious is the problem of crime and Z ry 
disruption in schools? . ° . / 


e How many schools, students, and 


‘ ¢ 
* “«s teachers are affected, in what ways, a 
and to what extent? _ 
4 a ® ’ 
7’ *@ When and where are the risks of crime and 
violence highest? . - : 
@ Who are the victims and offenders? 
e What are the attitudes and experiences of: 
the victims? 
e What factors are associated with violence _ : @ 
. and vandalism an schools? : 
@ -What measures are schools: aistng ta, reduce : 
or prevent crime? a iy 
@ What measures do principals, teachers, 
and students recommend? 
.@ What are the implications of this research, 
for policy? 
e 
D. SUMMARY ‘ 
~ 


Lag 
The following‘example illustrates: each overview component (abstract, 
background, esearch questions) discussed separately above, In this example, 
due to the compl ty of one project and the importance of its historical 


context, an extensive Baavenauna component was judged tai to an under- 


td] 


standing of the project. 


EXAMPLE: Project Overview : | Sy A 


- ABSTRACT 


COMPENSATORY EDUCATION STUDY 


’ — 
The National Institute of Education's (NIE) 
Compensatory Education Study was a comprehensive 
study undertaken at the request of Congress. ° : ; 3" : 
Its purpose was to gather information which would : : . 
help in developing and considering future legislation. 
After issuing a request for proposals. (RFP), we 
commissioned more than 35 substudies whose total cost 
was $15 million ove: a three-year period. Ten of those 
substudies investigated Title I funds allocation; two : 
surveyed the nature of compensatory education services; : 2 2 
seyenteen researched student development; ten investigated . 
the administration of Title I and state compensatory 4 
education programs. In addition to research in 
these four areas, NIE commissioned the development 
of a data archive. Its objective was to provide 
other researchers access to the compensatory. 
education data for secondary analysis purposes. . 
Six of the substudies were chosen for inclusion . %, 
in the archive. 


BACKGROUND a 
- 7 4 

The compensatory education concept stems from the 

recognition that children from disadvantaged back- , , 

grounds frequently do not enjoy the same educational 

benefits as their more affluent peers. Many dis- 

advantaged children attend schools in districts as 


, characterized by low overall revenues and/or high 


~ 


concentrations ef disadvantaged families. Such 
circumstances place special strains on ‘the schools 
and adversely affect the general educational 
development of pupils. Compensatory education is 
intended to ease these problems by providing dis- 


, advantaged children with additional services which 


will help them complete their education on more 
equal terms. 


The Elementary and Secondary Education Act (ESEA) was 

enacted in 1965; Title I of this act targeted’ large 

sums of money to "meet the special needs of educationally 

deprived children." Many hailed this legislation as 

an effective and central part of the War on Poverty 

because they believed it would, for the first time, 

open equal educational opportunities to children 

handicapped by their material circumstances. In en- _ ¢ 

acting Title I, Congress first sought to provide 7a: ™ 


additional financial assistance to school districts 

serving large numbers of students from low-income 

families and to schools enrol}ing the greatest number 

of such students. Second, Congfess sought to fund. ' 
special services for low-achieving children in schools « 
with low revenue levels. And third, @ongress intended ; 
Title I programs to contribute to the-cognitive, soeial, 
and emotional development of participating students. 


s 


. Ne 
Although Title I allocated large sums of money, it 
also included direct provisions limiting school 2 ; 
districts’ use of this money and requiring districts 

’ to perform. self-evalutions. These evaluations were to i 
be monitored by each state and analyzed in terms of cost ’ 


benefits. Thus, results could be fed into the United * 
States Office of Education's (USOE) decisionmaking i 
process and channeled back to the school districts. <t} 


In the years following the enactment of Title I, some of 

the early optimism faded because educators found weak- 

nesses in certain aspects of Title I administration and- : 
because information about the program was generally un- * © 
available. In practice, the unprecedented evaluation pro- ‘ 

vision failed to produce the information a ee to 

assist USOE in its decisionmaking. Federal and state 

enforcement of the evaluatidn was weak; schools resisted 

the requirement; the surveys performed lacked the hard 

data n sary to demonstrate success and were rarely 

read. Often, early national evaluations of Title I 

focused solely on children's academic performance, or, | 

more selectively, on reading programs only. In doing so, . 

these evaluations neglected the diverse programs also 
‘funded by Title I (e.g., math, science, health, counseling, 

and cultural enrichment). The scarce information concerning 
_ the nature of programs funded by Title I tended to describe ss 
only the incidence and distribution of various types of - 
programs. Few studies provided data concerning 
the cost of operating compensatory education ‘ 
programs and whether or not ‘the perEraas meeeTy - . 
worked. Le 


In aatution to this paucity of documentation 
on program design, effectiveness, and cost 
benefits, a growing debate surrounded the use .- 
of poverty measures as a cpiterion for Title I : 

eligibility. Advocates of\a change from poverty ‘ ‘ 
measures argued that since@the goal of 

compensatory education is to raise achievement 

levels, the most appropriate criterion is one 

which targets money directly to low-achieving 

children. Defenders of the status quo contended 

that the achievement criterion neglected the poor 


services were considered by some a mere substitu-. ‘ . r 
tion for state .compensatory funds. Others blamed - 

the federal guidelines for restricting districts 

from providing effective programs. A final 

criticism raised doubts about the basic intent 

of Title I: namely, that: the relationship between ‘ ; of 
poverty and educational performance had been mis- 

interpreted and misapplied. 


The Education Amendments of 1974 (PF 93-380) re- 

flected these dissatisfactions and sought to obtain } = 4 
information on ways to reauthorize Title IJ. The ‘ 
amendments pelhennee provisions which mandated NIE 


5 to 


NIE was directly responsible to Congress for the design 

and execution of the Compensatory Education Study. The i 
Study's framework was intentionally broad to encompass , ~ 
three elements common to all federal education programs: 


1. 


‘and subverted the fundamental intent of Title I: : 

to raise poverty levels by providing equal ed- 

ucational opportunity to the disadvantaged. Other 

criticisms of Title I were that districts mis- 3 

used their funds, either stretching them too 

thin or treating them as general aid and not. % 
really providing compensatory. eduéation services. : 

Some critics charged the State Educational. 

Agencies (SEAS) and the USOE Title I offices with 

inadequate monitoring of districts. Title I es > 


¢ 


c 


examine the fundamental purpose and effec- . - 
tiveness of compensatory. programs}; 


analyze ways of identifying children in greatest 
need of compensatory education; . 


develop alternatives for meeting whese children's : 
neefs; ~ 


consider the feasibility, costs, and consequences Salta 
of alternatives ‘for distributing compensatory \ 
education funds. E 


X 


Congress' objective ve improve education 
for children; , a 


funding and allocation procedures which a Ye 
Congress establishes in pursuit.of its | : ’ 
objectives; : 

operation of federal, state, and local 


agencies as Congress' agents in using 
federal funds to obtain its objectives. 
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RESEARCH QUESTIONS 


a 


To: fulfill Congress' mandate, NIE commissioned a 
total of 35 substudies, {including a National 

Survey Of Compensatory Education, special demon- 
stration projects concerned with alternative 8 
funds and allocation ‘procedures, and a number of 
detailed case studies. of particular aspects of ~« 
the Title I program. ' The 35 aabecudleu rachowented 
four foci of research activity. . ) . 


4; Fund Allocation OMe Sg . x 


+ 


‘ 


NIE's mandate to examine this topic was derined in < 
clauses (2),- (4), and (5) of Section 821(a) of. 4 
P-L. 93-380. This section called for jnvestigation. 
into the costs and feasibility of using alternative 
eligibility criteria for allogation of ‘Title I 
funds; current allodation, practice’; and assess- 
‘ments and simulations of the effects. of alternative 
criteria. Experimental project§ in which school 
districts selected their own eligibility criteria 
were also conducted. 


. 


2. Administration of Compensatory ehieseten 


clause (6) of Section 821(a) of P.L.. 93-380 ‘ 
most clearly expressed this areawf research 
activity. A survey of state regulations supple-. 
menting USOE Title I ‘Pegulations was conducted, 

as well as case studies describing the dmplementa- 
tion of regulations in a regionally representative- 
sample of states and districts. me 


a. Services to Students po, 


_ The federal- mandate to investigate this topic appeared 


in clauses of Section 821(a) of P.L. 93-380. 

Subsequent substudies included a survey of all Title 
’ I programs-to determine the nature and types 

of services presently provided. 


4, Student pawavogaait : a . 


‘ _f 
The NIE mandate to examine this area of compensatory 
edudation was contained in clauses (1) and (3) of © 
Section 821(a) of P.L. 93-380. Research efforts 
consisted of surveys of,compersatory activities to. t 
describe goals and methods employed by districts. 
‘ Data were synthesized and re-analyzed using materials 
from education agencies, the federal government, 
‘education organizations, and independent researchers. © 


~ 


‘ ‘+. ITI.. SUBSTUDY DESCRIPTIONS 
pay é 


After the PEREAge asa nee mas been described, the archivist provides 
a brief overview of the naicecnans between the substudies comprising the 


project. The pe of this anpreeh aA to inform the reader of how these _sub- 


studies come “toesthtr within the archive. Before this goal can be achieved, 
et 


the archivist must first organize the project {ito substudies. 
| Since the definition of what constitutes a substudy is not always obvious, 
we offer three guidelines to assist in this organizational task. 


1) Certain large-scale research projects actually consist 


of multiple independently-conducted studies; each independent 
study is a substudy. Usually, each of these substudies is 
undertaken by a different contractor (although this is not 
an essential defining characteristic); each has a different 


‘ i. focus, and each attempts to answer a specific set of research ; 


ba 
an questions. ~~ ‘ ¢ 
EXAMPLE: WHAT’IS A SUBSTUDY? © (1) 


The Compensatory Educati®n Study data archive con- ' 
sists of six independent substudies conducted by ‘i 

six different contractors. Each* substudy focuses 

cn a different aspect of Compensatory Education: 

Census Tabulations; Achievement Measures; Dencreteations 


State Administration Survey; National Survey; 
Instructional Dimensions Survey. Each also generated 
multiple data sets. , 


~- 


2) In song Seneee es data are collected from different target . ; 
populations aid generally SRLS sewanabelis Each target population's 


ae . data is a substudy, since a separate study ‘design was the basis 
a 
for selecting each type of target population and analyzing its 


response. 


®, 2 
oe ‘< 
‘EXAMPLE: WHAT IS A SUBSTUDY? (2) g 


r 
The Education Voucher Demonstration data archive 
consists of data collected longitudinally on 
students, teachers, parents, and schools. Sep- 
arate surveys were administered to each of these . 
target populations, and, while data from different 
files were sometimes merged, analysis activities 
: . = generally focused on each group as an independent 
entity. Therefore, the data youd be ican into 
four substudies. 


‘ 


3) A project operating under a single pcudy cae usually 
does not have component substudies, even if multiple data files were 
generated. The key aaa is whether the different 

iia eee were used to stoi substantially’ different 
aesniiok questions or to answer the same queacions from 
different perspectives. _ Sometimes, this distinction is 
quite difficult to make, since all projects have an overall 
research goal governing the collection of each data set. 
The issue is whether substantively different research 


, e issues were analyzed using each set of data. 


EXAMPLE: WHAT IS A SUBSTUDY (3) 


The Safe School Study collected data from principals, 
teachers, and students throughout the United States... 
Different data sets were collected from each group, 
e.g., teachers and students provided information on 
questionnaires and in follow-up interviews. However, 
all of the data sets were utilized to develop estimates 
of crime levels in schools in the U.S., to support 
various hypothesis of the causes of crime, and-to 
determine the most effective crime prevention measures. 
In this study, each data set was analyzed in light 
of the same research goal, so only one substudy 

- existed. 


r > 
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Each substudy is then detailed. Each substudy description 
consists of the following components: 
e title, 
* 
@ purpose, Ms -. 
. : * 
e design, - 
@ sampling procedures, . . ge ‘4 
ee © data analysis," — ele 
: ‘@ major findings, . : 
e fike descriptions. 
. at 
A. TITLE oi, 
Suibstudy titles need not be as detailed as project titles. Usually, 
> they are compact and contain no more than 15 words. However, some of the 
“ ‘ me. : 
conventions for the project title are followed in devising substudy titles, 
“¢ notably, the use of descriptive words, the placement of the most important . 
word first, and the avoidance of acronyms. 
EXAMPLE: SUBSTUDY TITLE 
Study Title: 
: Compensatory Education Study; 
ok National Institute of Education 


- Substudy titles: : °, 


1. Fund Allocation within Districts, Demonstration 
Studies 


2. Instructional Dimensions Study, 1976-1977 
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B. SUBSTUDY BACKGROUND AND PURPOSE. * 

The description of, ‘the substudy's purpose features the same 
components as the description of the sendanths purpose, except that it 
explains the relationship of the substudy to the overall purposes of thé 


project. It is usually one to twospages in length. 


EXAMPLE: SUBSTUDY BACKGOURND AND PURPOSE 


The Instructional Dimensions Study was funded by 
the National Institute of Education (NIE) as part 
of the Compensatory Education Study, a comprehensive 
research project conducted in response to a mandate 
by the U.S. Congress in the Education Amendments of 

. 1974, The Instructional Dimensions Study is designed 
both to gather data on program effectiveness in _ 4 
reading and mathematics and to compliment the NIE 
National Survey of Compensatory, Education, which 3 
describes the ways Title I funds are used nationally. 


The Instructional Dimensions Study is an in-depth 
assesssment of the relationships between selected 

= instructional constructs and students' achievement. 
These constructs are briefly described below.... 


C. SUBSTUDY DESIGN 


: ; ; 
The major question in this section is, What was investigated? It = 


is answered in a one- to’ four-paragraph description. 


EXAMPLE: SUBSTUDY DESIGN (1). 


In order to assesss the validity of our attitude 
measures and the idea that structural positions in- 
fluence attitudes, we first investigated the cross- 
sectional associations. between personal characteristics 
and attitude responses. Thirteen socio-demographic 
variables’ relevant to women's sex role attitudes were 
seen as falling into four categories. The four 
categories were then cross-tabulated with the socio- 
demographic variables. 
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EXAMPLE: SUBSTUDY DESIGN (2) . 


The effects of changes in demand for organizational 
services on various personnel categories were ex- 
amined. While the approach is intended to be 
general, the specifics of this theory are generated 
by characteristics peculiar to the organization 
under study, namely, school districts. After dis- 

d cussing the empirical findings, implications of the 
model for -the study of RTE ERTS dntensity were 
drawn. 


‘ 
‘A secondary question in the study design is, How was it investigated? | 


To answer this question, important variables in the study are identified, 


or unique indices, scales, or meaSures created or used in the study are noted. 


EXAMPLE: SUBSTUDY DESIGN (3) 


The analysis focused on the independent variables 

of marriage. and husband's characteristics. The de- * 
pendent varzables were all cultural variables con- \ 
cerning sex rolé ideology, P ‘Goad 


EXAMPLE: SUBSTUDY DESIGN (4) 


The Orshansky index was used to. determine poverty 
cutoff points. 


D. SAMPLING PROCEDURES 
? . * 


A good sampling description gives readers a prebis of the sample 


design and helps them to decide whether data in a substudy might be 


* 


nn 4, 


appropriate for Bnet papeeet research purposes. Therefore, sampling 


procedure is one of the most important elements for a researcher to examine ) 
, 

in substudy documentation. 
. i a" 


Each description of sampling. procedures includes descriptions of the 
universe, target, and obtained samples, and the data collection method. 


If appropriate, multiple source® of data and multiple surveys within a | 


- 
zc) 
Cy om 
° 
a 
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substudy are also briefly described. A detailed discussion of the actual 
4 
\ 
sampling technique or specific sampling design is unnecessary, since detailed 


documentation is provided tn.the file-level documentation. 


‘ 
4 


1, Target Sample and Universe Ls 
This section describes the population of respondents or subjects with- 

in the substudy data files. A brief description of the universe of subjects 

is given first,, followed by a description of the actual sample obtained for 

the data files. The reader is usually most interested in exactly who or 

what the data represent and the factors limiting the Sanble's use in analysis 

activities. This is an especially important consideration in large national 

sample surveys to prevent researchers from making unwarranted assertions 

based upon the poring Therefore, the archivist must alge any known limitations 

the sample imposes on potential analysis. ‘ : 

In addition, this section briefly describes the annpldoe method 


(random, stratified random, etc.),«response rates, and special procedures 
used to improve response. If multiple sets of-data were collected for a 


substudy, each sample is described. # 


2. Data Collection Methodology iF 


> 
‘ The method of collecting the data is explained here: mail survey, . & 


administered interview, observation, etc. Special characteristics entailed 
in the data collection process and the impact of these characteristics on 
analysis are briefly descrited. This section also notes when the data were. 


collected. i 


’ EXAMPLE: SAMPLE PROCEDURES (1) ‘ 


The primary data of the present study are from a 
_ questionnaire survey of 1,079 social scientists 
in the disciplines of anthropology, economics, 
8 political science, and psychology. ‘The total 


’ ; population was all social scientists; the target _— e 
: sample consisted of. the academic members listed 
in the most recent directories of the major pro- 
fessional associations representing the four | 
disciplines. 


A random sample of 500 was drawn from each discipline 
directory, and an initial 12-page form was sent 
in December, 1973. Four months later, after a 
second mailing of the questionnaire and a final y 
® follow-up letter, 1,079,usable responses had been ° 
obtained. The return rate was 54.04, a fairly 
common response level for college faculty. g 


EXAMPLE: SAMPLING PROCEDURES (2) , . - || 


1964 NORC College Seniors Study: Conducted by Fd 
the National Opinion Researsh Center at the 


; University of Chicago, this mail questionnaire 
survey was based on a national probability 

sample of approximately 42,000 women and men 
graduating from American arts and science colleges 
and universities in June, 1961. First questioned 
in 1961, the sample was remeasured for a second 
time in the fu.. of 1964. The analysis.used a 
randomly-selected 10% subsample of the total sample 
remeasured in 1964. The attrition rate for the, 
1964 data was 85% of the 1961 respondents. The 
response rate in 1961 was 95%. on 


| 


4 


EXAMPLE SAMPLING PROCEDURES (3) 
K 


Fifteen-Year Follow-up Survey 1970: Conducted by 
Bruce Eckland at the University of North Carolina, 
this survey sent mail questionnaires to a sample > 
of women and men drawn fifteenlyears earlier by 
the Educational Testing Services. The original 
respondents were high school sophomores and seniors 
at the time. Althougrk the sample was national ¢in 
scope, it was not a probability sample of all high 
school sophomores:and seniors. It can be viewed as 
‘either a probability sample of such persons with 
major exclusions (such as large cities) or asa. 
national nonprobability sample. About half of the 
original respondents to the Educational Testing 
Service study responded with usable questionnaires to 
al the Eckland follow-up (7,500 out of 15,000). | . 


¢ 
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E. DATA-ANALYSIS A 


The data waaluate section includes a brief explanation uf the statistical 
methods used in the study. Knowledge of the data analysis techniques used 
will enable future researchers to decide if additional types of analyses 
are indicated. However, detailed descriptions of the siaeaas performed 
in the original Study are of relatively limited value to future users of 
the archive. If these users desire more information about the statistical 
procedures used in the study, hay can cbtadin the final report of the original 


study which is cited in ‘the bibliography. 


EXAMPLE: DATA ANALYSIS (1) 


Commonality analysis was used to attribute portions of 

variations in achievement gain to.the various study 

model elements and to pretest performance. Costs 
«were also related to achievement gain. 


‘ ‘ 
EXAMPLE: DATA’ ANALYSIS (2) 


Achievement eligibility rates were estimated using re- . 
gression procedures corresponding to each of two 

models (Rank-9 and Ra&-8). The ‘independent variables 
included proportions of children in each location 

category, racial/ethnic group, and level of property 

status. ; ; a ? 


‘ 


EXAMPLE: DATA ss aguas (3) 


Two kinds of analyses were performed in this sub- 
study. Summary and breakdown tables (dedcriptive 
statistics) were used to describe the effects of 

the current poverty formula (e.g:, its distributional 
impact). Simulations using the Federal Education 
Finance (FEF) model were conducted to analyze the 
effects of the alternative povert} formulas. 


a 


F. MAJOR FINDINGS 
Only the major findings of the substudy are listed and summarized 


in one to three paragraphs. This summary includes a statement of whether 


c a ee ee 


¥ 


or not the substudy supported the hypothesis and a discussion of the 
relationship of the findings to the research questions described in the 


substudy purpose and design sections (B., D.). The summary should be a 


straightforward account of the study \findings and should not attempt to dis- 


cuss the significance of these findings, or their relationship to theory. 
nor to quoté percentages, tests of significance, or. other measures. Readers 
requiring such specific pine Sas can consult the final report citedsin 
i 

the bibliography. « ‘ a 
EXAMPLES: MAJOR FINDINGS . 

There is support for the interchangeability principle. 

Most of the combatant nations in the study experienced 

substantial increases in their rates of homicide. ; ‘ 

The increases were pervasive and occurred after large 

and small wars. . ae = 


The model accounts for a large proportion of the variance 
. in recidivism among first offenders. 


The number of friends who use marijuana is the best 
single predictor of whether the respondent will use 
marijuana. 

G. FILE DESCRIPTION . 

This section briefly describes each data file within a substudy. It 
informs readers of the type, scope, and scale of data in each file. 4 
Each file: description tells about . | 

e@ the type of data in the sae BEREDETES the reader to 

unexpected data and highlighting important or unusual = -> 
contents (e.g., this file is the only known source of 


national-weighted data on violence in schools broken a 


‘“ down by location within school); 
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¢ 


@e the data collection instrument used topreate the file; 
4 e the number of data items per subject; 


e the ‘number of subjects. 


“It is important to emphasize that rhe file description which occurs ' 
in the projecileved docundnkattan is a very brief overview of the data 
files associated with a eabn tule: The file descriptions should not be 
confused with-file-level documentation which appears in the separate 
file-level document. Recall that project-level documentation is re- 
searchers’ first look at a study; they are deciding whether or not to 

"a use the data ‘and the brief file description completes the picture. File- 
level documentation is read when the researcher has decided that the data 


meet his/her needs and, at that point, require more detail in order to 


cy 


design their secondary analys(fs. 


EXAMPLE: FILE DESCRIPTION (single-data file) 
Pd : : 
Semi-structured questionnaires were administered 
to a national random sample. The file:contains 
responses , from 1,504 persons on 316 variables. 
There were four decks or cards per respondent. 


EXAMPLE: FILE DESCRIPTION (multiple file substudy) 


Data for the longitudinal study comprise 37, 
. separate files. Thirteen instruments were used 
to collect data for year ohne; identical instruments 
were used for year two. Eleven of these instruments 
¢ were used againjin year three; however, the Math. - ™% 
and Language Arts Program Description Instruments 
were not used inthe third year. A brief discussion* 
- F of each Demonstration file follows. : 


‘Clas$room Roster ~ 


All homeroom teachers in-third- and fourth-grade © 
classrooms in each sample school were asked to 
use a self-administered form to describe the 
characteristics of students participating in com- 
pensatory activities. "These "rostered" students 
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represent the universe of students in the Demon- 
stration. Three files, one for each of the 
Demonstration’ years, are contained in this archive. 


1. The first year consists of data for 
51,029 students on 38 variables. 


2. The second-year file includes 38 
° variables for 47,859 students. 


3. The third-year file contains data 
for 45,529 students in 38 variables. 


i : Principal School and Background Information y 


All elementary school principals in the Demonstration 
districts were asked to complete a self-administered 
questionnaire to describe the compensatory program(s) 
operating in their schools. These district/school- 
level data are contained-in three files. 


. 


1, The first-year responses of 596 principals 
to 348 background variables are included in gi 22 
this file. : 


. 


‘2. The second-year principal responses representing 
152 background variables on 474 schools reside 
in this bases 


3. The third-year rile contains 284 variables from 
367 princifa@ls about their schools’ programs. 


Principal Supplement 1, Title B.S.E.A. ‘ - 


These data are the self-administered questionnaire re- 
of sponses of all principals of schools receiving Title I. 
The data contain descriptions of Title I services pro- 
vided in each school. Principals were interviewed in : 
each of the study years; thus three files for each 
- administration have been archived. ° 
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IV. APPENDICES 


A. BIBLIOGRAPHY 
Two types of. bibliographies, each divided into two sections, may 
appear in the appendices. The first epiie is an overall project bibliog- 
nphyi the se d type is “ substudy bibliographies. Each of these ” 
bibliographies may be broken down into two sections. One section ‘ 


- 


lists reports generated by the study or substudy, that is, reports 
that were products of primary or secondary analysis of the study's . 
data. The second section contains references to reports, articles, 


and papers which provide background information, such as enabling ~ 


q legislation or discussions of the issues or.topics being researched. 


B. GLOSSARY . 
The purpose of the glossary is to demystify project terns for those 
persons who will conduct research long after the original research team dis- 
bands. Secondary analysts using data from a project may come from a wide 
variety of academic disciplines, private research organizations, or policy 
“groups. In many projects, a number of key, terms are used again and again, 
\ "and take on new and special meanings that are eventually taken for granted 
\ 


* by persons working on the project. Such terms are a familiar shorthand for 


‘those directly involved in a project; to a secondary analyst, however, they 


. may be vague, meaningless, or likely to be misconstrued. For this reason, 
a glossary of terms used in each substudy is incorporated in the appendices 


to the projecvu-level documentation. The glossary is alphabetized, sometimes 
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several pages in length, and indludes acronyms, abbreviations, and special 


terms referring to 


@® government agencies, research consortiums, or research 


organizations; 


e enabling legislation, administrative units, and programs 


4 


administration; 


e study variables; 
@ data analysis models; ’ 


@ other concepts. 


The glossary also includes more familiar words whose meanings are "new" 


t . 
because of the way they are used‘in the project. 


Example: Acronyms for government agencies, research consortiums, or 
research organizations. 


1. AAAS. American Association for the Farvsnoenent of Science. 
2. APA. American Psychological Association. 
: 3. DHEW. Department of Health, Education/and Welfare. 


4, NIE. The National Institute of Education. 


™ 
Examples: A¢ronyms and terms referring to enabling legislation, 
administrative ynits, and program administration. 


ESEA. The.Elementary and Secondary Education Act ‘ 
(ESEA) of. 1965. 

C.E. Refers to the Compensatory Education Study \ 
as a whole. 


Section 821. An educational amendment instructing the 
National Institute of Education to conduct studies 
evaluating the ESEA Title I compensatory education 
programs. 


¢- 


_ Examples: Acronyms and names for variables in the study; operational 
terms. 


IDS. Instructional Dimensions Study, a substudy of the 
Compensatory Education Study which assessed the separate 
and combined effects of five sets of classroom processes: 
opportunity, individualization, instructional events, . 
motivational processes, and teacher background. 


‘ study, the measures of classroom climate and the incentives 


Examples: 


* 


Matching by mastery. In the Instructional Dimensions 
substudy, the presence and use of aaa a 
method to assess maytery. 

i 


okivatdonel processes. In the’ Instructional Dimensions 


teachers” provided students for learning. 


Terms for data analysis and models. 


‘D! The index of dissimilarity which measures segregation. : 


dtis anff Beverly Duncan use the term to refer to ecological 
segregation. 


‘ 


Pullout instruction. Refers to the teacher actually taking 


the student out of the classroom setting for individualized 
instruction vs. regular instruction where the student is 
given instruction with the combined class. 
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